Information Systems Education Journal (ISEDJ) 


9(3) 
August 2011 


Using Undergraduate Information Systems 
Student Epistemic Belief Data in Course Design: A 
Research-based Approach to Improve Student 

Academic Success 


Samuel S. Conn 
samuel.conn@kysu.edu 

John English 
john.english@kysu.edu 

Fred Scheffler 
fred.scheffler@kysu.edu 

Kentucky State University 
Frankfort, KY 40601 USA 

Simin Hall 
simin.hall@vt.edu 

Virginia Polytechnic Institute and State University 
Blacksburg, VA 24061 USA 


Abstract 

In this report the authors detail a baseline study involving use of epistemic belief data to enhance 
academic success collected from an undergraduate student population enrolled in an Information 
Systems undergraduate degree program. Based on an existing line of inquiry, student epistemic belief 
data were collected and analyzed to determine student perception of knowledge and levels of self¬ 
regulation and self-efficacy. Indicators were determined through item analysis and evaluated for use 
with an existing epistemic belief profile rubric. Working in concert with course developers, strategies 
for altering approaches in instructional design, pedagogy, and assessment based on student epistemic 
beliefs were determined. Researchers from institutions of similar composition can benefit from findings 
of this study. Moreover, strategies for altering a student population's trajectory toward improved 
academic success were an outcome of this study and included application and analysis of: (a) student 
epistemic belief data and its role in higher education, (b) relationships between epistemic beliefs and 
student academic success, and (c) a methodology for improving student academic success via 
research-based instructional design, pedagogy, and assessment. 
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1. PROBLEM STATEMENT 

The problem examined in this study involved use 
of epistemic beliefs in course design. Student 
epistemic beliefs, juxtaposed against the theory 
of knowledge, degrees of student self-regulation, 
and cognitive development theories, can be used 
to design more efficacious courses if an 
integrative methodology is applied. Creation of a 
course development methodology involving use 
of student epistemic beliefs is problematic. 
Relating theory of knowledge, degrees of self¬ 
regulation and self-efficacy, and cognitive 
development theory as dimensions to construct 
a student population profile and use of student 
epistemic belief data to position a given student 
population within the construct is complex. 

In this investigation, the authors illustrated the 
congruence of theory of knowledge, degrees of 
student self-regulation, and a cognitive 
development theory as a framework for 
determining appropriate course instructional 
design strategies. A rubric involving student 
epistemic belief profiles was applied in response 
to a prescriptive-diagnostic approach (Schunk, 
1983). Through this research and case-based 
study the authors wanted to know: a) what are 
students' epistemic beliefs regarding knowledge; 
b) what are students' epistemic beliefs regarding 
self-efficacy; c) what are students' epistemic 
beliefs regarding self-regulation; and d) what 
are students' epistemic beliefs regarding 
instruction? Once determined, the researchers 
constructed a profile for the student population 
based on epistemic belief data. 

The profile was used to establish a baseline for 
pedagogy and assessment strategies; using an 
existing rubric, a strategy for trajectory to 
higher levels of epistemic belief was plotted. The 
authors posit that course designers and 
developers can apply the design elements to 
achieve a course of instruction in harmony with 
an existing student population's epistemic 
beliefs, or to construct a pathway to alter 
epistemic beliefs toward an optimal goal of 
constructivism, commitment and constructed 
knowledge, and high levels of self-regulation 

2. LITERATURE REVIEW 

Philosophy addresses the nature and rationale 
for human knowledge through an area of 
concern referred to as epistemology. According 
to Hofer and Pintrich (1997), individual 
epistemology, or epistemic beliefs, involves 
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one's beliefs regarding the nature of knowledge 
and knowing. Early theorists (e.g., Perry, 1970; 
Pintrich and Schunk, 2002; Schoenfeld, 1985; 
and Hofer and Pintrich, 2002) promoted the idea 
that epistemic beliefs alter students' learning 
strategies, problem solving capabilities, 
comprehension, and achievement of learning 
outcomes. Major theories developed by 
educational psychologists such as Buehl, 
Alexander, and Murphy (2002), Hofer and 
Pintrich (1997), Muis, Bendixen, and Haerle 
(2006), Piaget (1950), and Schommer (1990) 
incorporate and apply some element of student 
epistemic beliefs. 

As a result, epistemic beliefs are deemed to 
influence learning, motivation, and cognition. 
Integrative studies of student epistemic beliefs 
with other learning theories and models have 
evolved over the past few decades, e.g., Bloom, 
Engelhart, Furst, Hill, & Krathwohl (1956); 
Ryan(1984a, 1984b); and Muis (2007). 

Based on Hofer and Pintrich (1997), epistemic 
beliefs affect four dimensions of knowledge: (a) 
certainty of knowledge, (b) simplicity of 
knowledge, (c) justification for knowing, and (d) 
source of knowledge. According to Schommer 
(1990), certainty of knowledge is reflected as a 
continuum with a belief that knowledge is 
absolute and unchangeable on one end as 
opposed to a belief that knowledge is tentative 
and evolving on the other end. Moreover, 
simplicity of knowledge is illustrated as a 
continuum with a belief on one end that 
knowledge is defined as isolated, unambiguous 
chunks as opposed to a belief that knowledge is 
defined as highly interrelated conceptualizations. 

According to King and Kitchener (1994), 
justification for knowledge also can be depicted 
as a range where knowledge requires no 
justification to where knowledge is constructed 
and critically refined and reevaluated. Based on 
Kuhn (1993), epistemic beliefs influenced by 
"source of knowledge" can range from total 
reliance on and acceptance of authoritative 
experts, to critical evaluation of expert 
knowledge. 

In accordance with Muis (2007), two high-level 
architectures exist with respect to epistemic 
beliefs, one motivated by a developmental 
perspective and one motivated by a 
multidimensional perspective. Perry's (1970) 
work illustrates a developmental perspective in 
defining a student's initial view of knowledge 


©2011 EDSIG (Education Special Interest Group of the AITP) 
www.aitp-edsig.org /www.isedj.org 


Page 10 



Information Systems Education Journal (ISEDJ) 


(absolutism/objectivism), a progression to a 
more advanced view of knowledge 
(multiplism/subjectivism), and progressively the 
highest view of knowledge 

(evaluativism/objectivism-subjectivism). In 
contrast, Hammer and Elby (2002), Hofer and 
Pintrich (1997), and Schommer (1990) proposed 
multidimensional frameworks, where 
incremental, non-sequential knowledge 
dimensions assemble to form and represent 
knowledge. 

Muis (2007) established a relationship between 
epistemic beliefs, self-efficacy, and self- 

regulated learning. Investigations, e.g., Ryan 

(1984b); Schoenfeld (1983, 1985); Schommer 
(1990); and Hofer (2000), have determined a 
relationship between epistemic beliefs and levels 
of meta-cognition. According to Knight and 

Mattick (2006), researchers increasingly are 
finding a relationship between epistemic beliefs 
and disciplinary domains, i.e., epistemological 
beliefs are discipline specific. In effect, student 
epistemic beliefs can be juxtaposed with known 
theories and models of learning to establish 
baselines for given populations defined by 
discipline or content domain. 

Pintrich and Schunk (2002) demonstrated that 
successful self-regulated learners possess higher 
levels of motivation (personal influences), apply 
more effective learning strategies (behavioral 
influences) and respond more appropriately to 
situational demands (environmental influences). 
In addition, Hofer and Pintrich (1997) 
hypothesized that epistemic beliefs affect 
achievement mediated through self-regulated 
learning. 

Schunk (1995) defined self-regulated learning as 
"learning that results from students' self¬ 
generated thoughts and behaviors that are 
systematically oriented toward the attainment of 
their learning goals" (p. 125). Moreover, 

Bandura (1986) showed that self-efficacy beliefs 
impact performance because these beliefs 
represent people's perception of their 
capabilities to perform a task at designated 
levels. These researchers have provided 
empirical data on causal or correlation 
relationships between self-efficacy and epistemic 
beliefs and self-regulated behaviors and 
performance in subjects such as mathematics 
(Pajares & Miller, 1994; Schommer et al, 1992; 
and Schunk, 1981, 1984). 
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Social constructivism (Pajares, 2002) provided a 
basis for this case study's course construction 
recommendations and related instructional 
strategies. Social constructivism suggests that 
the exchange of critical feedback among peers 
as well as from the instructor can encourage 
students to modify their work. Learners engaged 
in a collaborative problem solving process 
receive feedback and comments from peers and 
from the teacher on related steps of planning, 
implementing, and executing problem solving 
processes rather than only receiving feedback 
from the instructor on their performance. 

Feedback is an important consideration because 
it requires transfer of knowledge and therefore 
represents students' gain in problem solving 
(Clark & Mayer, 2003). In particular, feedback 
from peers may push students to perform higher 
level cognitive functions (Schoenfeld, 1983). 
Furthermore, social cognitive theory posits 
reciprocal interactions between behaviors, 
cognitions, and environmental variables 
(Bandura, 1984) can enhance self-efficacy as it 
relates to problem solving skills. Feedback from 
peers and instructor are environmental variables 
as well as the modes of course delivery that can 
influence student confidence as it relates to the 
acquisition of problem solving skills (Schunk & 
Pajares, 2002). 

Moreover, social cognitive theories posit as 
possible the design of an educational experience 
such that learning occurs and is enhanced as a 
result (Marra & Palmer, 2004). Designing a 
course such that student learning takes place 
requires examining student epistemic beliefs, 
how feedback is utilized during learning, as well 
as student perceptions of teaching and learning. 
For example, students who require and expect 
more instruction do so in part because of their 
epistemic beliefs regarding the nature of 
knowledge and knowing. Research has shown 
that epistemic beliefs affect how students 
approach learning tasks (Schoenfeld, 1983), 
monitor comprehension (Schommer et al., 
1992), and plan for solving problems and carry 
out those plans (Schommer, 1990). 

Course design can be used to enhance 
collaboration and feedback through active 
engagement with materials and collaboration 
with peers and instructors. Online resources 
such as chat, discussion forum, blog, and wiki 
can play an active role in facilitating 
collaboration and feedback. One appeal of 
asynchronous technologies is that learners can 
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access materials, complete assignments, 
participate in discussions, and take exams 
according to schedules that they themselves 
determine. Hypermedia learning environments 
offer particular advantages to learners who are 
inherently self-directed learners (Mayer, 2002). 

However, at many institutions the current 
population taking courses consists of traditional 
undergraduates. These students typically require 
and expect more structure and instruction 
(Ravert & Evans, 2007). Many students, 
particularly those with low motivation, 
achievement, and self-regulation are unwilling to 
do mindful work, such as executing higher level 
cognitive processes that are involved in 
scholastic work (Report to Congress, 2004). 

3. METHODOLOGY 

Researchers in this study utilized a mixed- 
method approach in collection of qualitative and 
quantitative data (Creswell & Clark, 2007). A 
case study methodology was used to collect 
relevant qualitative data regarding the subjects 
of the study, undergraduate students in their 
first year of study. Likert scale data were 
collected; survey instrumentation was used to 
collect quantitative data involving dimensions of 
student epistemic beliefs. Based on an item 
mean analysis of the quantitative data, the 
student population was identified by level of 
epistemic belief: simple, moderate, 

sophisticated. 

Moreover, data analysis included standard 
Pearson Correlation Co-efficient (r) and Factor 
Pattern analysis using Eigenvectors and Varimax 
rotation method. In accordance with existing 
lines of research regarding epistemic belief data, 
the researchers determined the most efficacious 
framework for instructional design, pedagogy, 
and assessment to improve student success in 
Information Systems coursework. 

In this mixed-method investigation a qualitative 
case study methodology was applied and 
supported by quantitative data from an 
undergraduate Information System student 
population. The diagnostic- prescriptive 
framework involved a logic chain beginning with 
collection of data from a specific population 
regarding student epistemic beliefs. Data 
analysis and conventional heuristics yielded 
prescriptive indicators of placement of the 
sample student population relative to a three- 
dimensional framework (Figure 1) constructed in 
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concert with accepted learning theories and 
models (i.e., developmental perspective models 
and multidimensional perspective models) and 
the social cognitive theory of self-regulated 
learning. 

Based on the three-dimensional framework, a 
rubric of 27 design elements for course 
construction was applied. Course design 
elements in the rubric accommodate Bloom's 
hierarchy of cognitive development, synchronous 
and asynchronous pedagogical strategies, and 
assessment of learning achievement based on 
level of epistemic belief (Hannafin & Hill, 2007). 



Figure 1: Framework for determination of 
learner epistemic beliefs profile 
Source: Conn, Hall, and Herndon (2010) 


As illustrated in Figure 1, three axes 
representing continuums based on Perry's 
scheme, theory of knowledge, and levels of self¬ 
regulation were abstracted as a cube with 27 
distinct co-ordinate dimensions: xl, yl, zl 
through x3, y3, z3. This three-dimensional 
modeling technique was used to identify specific 
characteristics and profiles for a given 
population of learners. To create the x-axis 
(Figure 1), Conn, Hall, and Herndon (2010) 
grouped Perry's (1970) nine "positions" relative 
to knowledge and learning into three groups: 
dualism, relativism, and self¬ 

affirmation/commitment. Dualism includes 
Perry's positions of basic dualism, pre-legitimate 
multiplicity, and legitimate but subordinate 
multiplicity (Marra, Palmer, & Litzinger, 2000). 
Relativism includes full or legitimate multiplicity, 
contextual relativism, and foreseen 
commitment; self-affirmation and commitment 
includes commitment within relativism. 
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Based on items means from the data collected, 
the sample learner population was described 
based on the three-dimensions. For example, a 
sample population located in the xl, yl, zl 
dimension (Figure 2) would characteristically be 
described by tendencies toward dualistic 
knowledge and learning, absolute knowledge, 
and low levels of self-regulation. This non- 
optimal position would indicate epistemic beliefs 
of the lowest order (simple), thus requiring 
instructional design and pedagogy consistent 
with initial levels of cognition, student 
motivation, and self-efficacy (Schunk & Pajares, 
2005). Any shift in the dimensional positioning 
would indicate movement in a positive direction, 
where mutual recursion or other reciprocal 
relationship may be evident. 



xl.yl.zl 

Learners epistemic beliefs 
characterized by: (a) low self-efficacy 
and low self-regulation; (b) total reliance 
on teacher as the sole authority and 
source of knowledge; and (c) only right/ 
wrong answers or one solution. 


Figure 2: Framework positioning for non- 
optimal learner epistemic beliefs profile 
Source: Conn, Flail, and Flerndon (2010) 

In another example (Figure 3), a sample 
population located in the x3, y3, z3 dimension 
would characteristically be described by 
tendencies toward self-affirmation/commitment, 
high levels of meta-cognition, an ability to 
construct knowledge through collaboration, 
synthesis, and evaluation, and a high level of 
self-regulation. This optimal position would 
indicate highly evolved epistemic beliefs 
(sophisticated) that could accommodate 
instructional design and pedagogy consistent 
with advanced cognition and self-efficacy 
(Pajares & Kranzler, 1995). 

With respect to phenomena involving reciprocity 
between axes in the framework, a learner 
population with a cognitive ability to construct 
new knowledge and act as a source of 
knowledge would demonstrate higher levels of 
self-regulation. Conversely, learner populations 
with higher levels of self-regulation would 


possess attitudes and epistemic beliefs to 
construct knowledge, use interdisciplinary 
approaches in problem solving, and appreciate 
and incorporate multiple perspectives in the 
creation of new knowledge. 

4. ANALYSIS OF SURVEY DATA 

An existing survey instrument was utilized to 
measure three dimensions: a) students' 
perception of knowledge and knowing, b) 
students' level of self-regulation, and c) 
students' perception of self-efficacy. Thirteen 
questions measured students' perception of 
knowledge and knowing, including perceptions of 
instruction. Eight survey questions related to 
level of self-regulation. Item responses for these 
dimensions were obtained using a 5-point Likert 
scale ranging from 1 (Strongly Disagree) to 5 
(Strongly Agree). Fifteen survey questions 
related to self-efficacy and asked participants 
how confident they were in solving various 
problems and their self-confidence as it related 
to stating what is known or what is to be 
determined after reading a sample problem 
statement. 



Figure 3: Framework positioning for optimal 

learner epistemic beliefs 

Source: Conn, Hall, and Herndon (2010) 


Response options involved a five-point Likert 
scale ranging from 1 (no self-confidence) to 5 (a 
high level of self-confidence). The instrument 
scored a reliability coefficient of 0.86 in this 
baseline study. Mean scores were computed for 
each item on the survey. Factor analysis was 
used to develop three scales for the tree 
constructs measured in the survey. Chronbach 
alpha scores were used to ensure reliability for 
the three scales. 
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In this study, the sample population (N = 28) 
consisted of undergraduate Information Systems 
students composed of 14% freshmen, 21% 
sophomore, 36% juniors, and 29% seniors. 
Respondents were 21% female and 79% male. 

Analysis of Perry's Scheme Sub-scale Data 

For item one, students responding with a 4 or 5 
(75%) indicated that they agreed or strongly 
agreed that "A good college instructor often 
brings up questions that have more than one 
answer." Therefore, as believed by the student, 
good (i.e., effective) instruction promotes 
multiple answers to questions. As a result, a 
moderate item mean score (3.85 with SD=.854) 
indicates a preference for instruction originating 
from multiple sources. The second item, as an 
indication of tendency, where a response of 4 or 
5 (96%) indicated they agreed or strongly 
agreed that "College instructors should present 
various ideas on an issue", calculated to a mean 
of 4.42 with SD=1.09. The students were not 
skeptical of multiple answers to a single 
question, thus their tendency is toward hearing 
all arguments and ideas surrounding an issue. 
Item three confirms this conclusion where 
students responding with a 4 or 5 (39%) 
indicated that they agreed or strongly agreed 
that "It's not necessary for the instructor to 
answer all of my questions I ask in class; fellow 
students can often do it instead" and calculated 
to a mean of 3.07 with SD = .324. 

Further confirmation is seen in item five where 
students responding with a 4 or 5 (86%) agreed 
or strongly agreed that "In a good course I 
would learn as much from fellow students as I 
would from the instructor" and calculated to a 
mean of 4.00 with SD=.400. Item seven scored 
consistently with a mean of 2.67 (SD=.434) 
when the sample population responded to the 
statement "In class, I want other students to 
answer the questions I ask instead of the 
instructor answering my question." Of students 
responding with a 4 or 5 (25%), 50% disagreed 
or strongly disagreed with answers coming from 
an alternative, convenient source of knowledge 
(i.e., classmates), indicating emergence to 
constructivism and away from absolute 
knowledge. 

In concert with movement away from absolute 
knowledge tendencies, the percentage of 
students who agreed or strongly agreed with "I 
like it when an instructor brings up a question 
that he or she doesn't know the answer to" 
evaluated to 40%, indicating a transition away 
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from the belief that instructors are authority 
figures who should know all the answers. 


Table 1: Perry's Scheme Sub-scale (x axis) 


Mean 

STDEV 

Ql. A good 
college 

instructor often 
brings up 
questions that 
have more than 
one correct 

answer. 

3.85 

.854 

Q2. College 
instructors 
should present 
various ideas on 
an issue. 

4.42 

1.09 

Q3. It's not 
necessary for 
the instructor to 
answer all of my 
questions I ask 
in class; fellow 
students can 
often do it 
instead. 

3.07 

.324 

Q4. I like it 
when an 
instructor brings 
up a question 
that he or she 
doesn't know 
the answer to. 

3.07 

.400 

Q5. In a good 
course I would 
learn as much 
from fellow 
students as I 
would from the 
instructor. 

4.00 

.969 

Q6. I usually like 
it when my 
instructor 

answers a 
question with "it 
depends"and 
follows this 
statement with a 
discussion of the 
topic. 

3.82 

.666 

Q7. In class, I 
want other 
students to 
answer the 
questions I ask 
instead of the 
instructor 
answering my 
question. 

2.67 

.434 
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For item six, 68% of respondents agreed or 
strongly agreed that they "usually like it when 
my instructor answers a question with 'it 
depends' and follow this statement with a 
discussion of the topic"; calculated as a mean of 
3.82 (SD = .666). As a result, the population 
generally accepts that knowledge is contextual, 
indicating relativistic thinking. 

In sum, the scores reflected from student 
responses in the Perry's Scheme Sub-scale (x 
axis) indicate an evolving preference away from 
the instructor as an authoritative singular source 
of knowledge, and outlier tendencies toward 
multiplicity of knowledge and knowing. The 
overall item mean for this Sub-scale calculated 
to be 3.56 (SD=.694) indicating the population 
holds relativistic beliefs with emerging self- 
affirmation and commitment tendencies. 

Analysis of Absolute Knowledge Sub-scale 
Data 

For item eight, students responding with a 4 or 5 
(35%) indicated that they agreed or strongly 
agreed that "If I heard an instructor say 'we 
don't know the answer to that', they would 
worry about taking a class from him/her." As 
believed by the students, the instructor should 
not know all, and 39% indicated every question 
has one correct answer. Moreover, the items 
together indicate a tolerance for knowledge that 
is transient or evolving. As a result, a moderate 
item mean score (2.82 with SD=.464) indicates 
movement away from a preference for absolute 
knowledge and knowing. 

Flowever item nine, where students responding 
with a 4 or 5 (14%) indicated that they agreed 
or strongly agreed that "An instructor who says 
'nobody really knows the answer to that' is 
probably a bad instructor", illustrates a tendency 
toward evolving and transient knowledge. 
Combined, item analysis indicates students do 
not worry if questions have no single answer, 
and have an emerging acceptance that 
knowledge does not have to be absolute, 
dualistic, and unambiguous. 

Responses to Item 10 indicate a moderate level 
of perception of knowledge (item mean of 3.03 
with SD=.473) and evolving relativistic 
tendencies. Of students responding with a 4 or 5 
(39%) stated that they agreed or strongly 
agreed that "There is one right answer for most 
questions and a good instructor knows it." 
Nearly half (43%) of the population disagreed or 
strongly disagreed, indicating strong movement 
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toward experiential knowledge as a basis for 
learning. This conclusion is supported by item 11 
where 32% of respondents agreed or strongly 
agreed that "A good instructor gives facts and 
leaves theories out of the discussion." The mean 
for this item calculated to be 3.17 with 
SD = .716. 

Moreover, item 12 scores lag support and 
indicate a strong tendency toward absolute 
knowledge and low self-regulation. Those 
responding who indicated a 4 or 5 (82%), 
agreed or strongly agreed with the statement 
"An instructor's main job is to make sure I learn 
the course material"; however, the population 
generally indicates a preference for experiential 
knowledge that is evolving and transient in 
nature. 


Table 2: Absolute Knowledge Sub-scale (y 
axis) 


Mean 

STDEV 

Q8. If I heard an 
instructor say 
"we don't know 
the answer to 
that" I would 
worry about 
taking a class 
from him/her. 

2.82 

.464 

Q9. An instructor 
who says 
"nobody really 
knows the 
answer to that" 
is probably a bad 
instructor. 

3.39 

.696 

Q10. There is 
one right answer 
for most 
questions and a 
good instructor 
knows it. 

3.03 

.473 

Qll.A good 
instructor gives 
facts and leaves 
theories out of 
the discussion. 

3.17 

.716 

Q12. An 

instructor's main 
job is to make 
sure I learn the 
course material. 

2.07 

.440 


Analysis of Self-regulation Sub-scale Data 

For item 13, students responding with a 4 or 5 
(50%) indicated that they agreed or strongly 
agreed that "It is my own fault if I don't learn 
the material in a course." As believed by the 
sample, half take responsibility for their own 
learning. As a result, a higher item mean score 
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(3.42 with SD=.448) indicates a preference for 
experiential knowledge and internal motivation 
to learn. This conclusion is supported in 
responses to item 14 where students responding 
with a 4 or 5 (11%) indicate a decrease in 
accepting a relationship between level of effort 
and time-on-task and achievement of learning 
outcomes. A lower mean (2.39 with SD=.398) 
indicates lower levels of self-regulation in the 
population. Avoidance of increased level of effort 
and time-on-task is generally indicated as a 
contributing factor to low levels of self¬ 
regulation. For this sample, 29% agreed or 
strongly agreed in item 15 with the statement 
"Often when I am bored, I like to study"; a 
mean calculation for this item was determined to 
be 2.60 with SD=.330. 

With respect to focus and attention as indicators 
of level of self-regulation, 25% of the sample 
agreed or strongly agreed in item 16 with the 
statement "During the time I am in class, I often 
miss important points because I am thinking of 
other things". Mean response to item 16 
calculated as 2.53 with SD= .539. In concert 
with this response, 33% of the sample agreed or 
strongly agreed that "I often feel so lazy or 
bored when I study that I quit before I finish 
what I planned to do". Item 17 mean calculated 
to 2.64 with SD = .595. Moreover, ability to stay 
focused also supported low to moderate levels of 
self-regulation as 36% of the sample (a mean of 
2.78 with SD=.480) indicated agreement or 
strong agreement in item 18 that "I often find 
that I have been reading for class but don't 
know what it was all about". Student response 
to item 19 indicated that 46% agreed or strongly 
agreed with the statement "I find it hard to stick 
to a study schedule"; item mean calculated as 
3.10 with SD=.473. 

These scores support the conclusion that the 
sample population demonstrates low to 
moderate levels of self-regulation. Item 20, the 
final item in the self-regulation sub-scale, 
evaluated in support of low self-regulation and 
absolute knowledge as 43% of students ( a 
mean of 3.07 with SD=.406) surveyed agreed or 
strongly agreed with the statement "In most 
cases, I can learn the course material whether 
the instructor teaches it well or not". 

Based on the literature, the authors posit that 
students with overall higher levels of epistemic 
belief exhibit more self-regulated behaviors, 
have less preference for absolute knowledge, are 
able to evaluate multiple views and approaches 
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toward solving problems and learning theories, 
do not depend on instructors as a singular 
source for learning, do not think instructors are 
authority figures and are the only source of 
knowledge, and enjoy and willingly contribute to 
peer discussions and collaborative learning. 


Table 3: Self-regulation Sub-scale (z axis) 


Mean 

STDEV 

Q13. It is my 
own fault if I 
don't learn the 
material in a 

course. 

3.42 

.448 

Q14. If I don't 
understand the 
course material, 
it is because I 
didn't try hard 
enouqh. 

2.39 

.398 

Q15. Often when 

I am bored, I like 
to study. 

2.60 

.330 

Q16. During the 
time I am in 
class, I often 
miss important 
points because I 
am thinking of 
other thinqs. 

2.53 

.539 

Q17. I often feel 
so lazy or bored 
when I study 
that I quit before 

I finish what I 
planned to do. 

2.64 

.595 

Q18. I often find 
that I have been 
reading for class 
but don't know 
what it was all 
about. 

2.78 

.480 

Q19. I find it 
hard to stick to a 
study schedule. 

3.10 

.473 

Q20. In most 
cases, I can learn 
the course 
material whether 
the instructor 
teaches it well or 
not. 

3.07 

.406 


A different set of interventions, course design 
elements, and instructional strategies would be 
indicated: (a) if students believed knowledge 
consists of isolated facts and they did not 
engage in transfer or considered relationships 
among facts, (b) if students view instructors as 
the only possessor of knowledge, and/or (c) if 
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students were not prepared developmental^ to 
engage in peer collaboration to solve problems 
and create knowledge. 

Student survey data, organized in Tables 1-3, 
were related to Figure 1, a framework to profile 
learner epistemic beliefs, via item means. 
Indicators for the framework x-axis, a range 
from dualism to self-affirmation and 
commitment, are seen in Table 1. Further, 
indicators for the framework y-axis, a range 
from absolute knowledge to constructivism, are 
seen in Table 2. Finally, indicators for the 
framework z-axis, a range from low self¬ 
regulation to high self-regulation, are seen in 
Table 3. Following a means procedure, item 
means for each table were calculated (Table 4). 
The item means were projected into three linear 
ranges that reflect one of three axes positions in 
the framework: 1, 2, or 3; defined as follows: 


Item Means -> Axis Position 
0.00 - 1.67 1 

1.68 - 3.34 2 

3.35 - 5.00 3 


Prescription for this Information Systems 
student population was achieved by scoring each 
variable item mean to an axis position according 
to the (relative) indicators defined in the 
framework. Based on the item means for each 
variable (Table 4) and the item means to axes 
positioning, this case study population of 
students was defined within the framework as 
x3, y2, z2. The mean positioning of x3 indicates 
the learner population is characterized by 
relativistic tendencies emerging toward self¬ 
affirmation/commitment. 


According to Perry (1970), this learner 
population has moved past views that answers 
are either right or wrong and problems have 
only one solution, and have begun to adopt a 
view that knowledge is contextual and transient. 
The learner in this population is beginning to 
accept himself/herself as a legitimate source of 
knowledge and generally does not consider the 
teacher to be the absolute authority or source of 
knowledge. 

Positioning of the learner population as y2 
comes as a result of variable means of 2.78 
from Table 4. In this position the learner is 
characterized as still having some preference for 
dualistic, binary thinking, but is fully capable of 
relativism. To advance trajectory, pedagogy and 
assessment should involve reflecting on previous 


experience, collaboration with peers, creation of 
mental models, and application of cognitive 
schema; learners in this coordinate position can 
begin to learn how to construct new knowledge 
if given appropriate tools and directions. 
Learners in the y2 position also exhibit a 
predisposition toward experiential learning, can 
manipulate a body of knowledge to abstract 
salient points, and can visualize simple abstract 
concepts and models. Moreover, learners in this 
position can incorporate nascent experiences 
into an existing cognitive framework or 
reference and accommodate new theories, 
concepts, and schema (Perry, 1981). Learners at 
this higher cognitive level also are transitioning 
from passive to active learners and generally 
learn by doing. 


Table 4: The MEANS Procedure 


N 

Mean 

STDEV 

Table 1: 

Perry's 
Scheme Sub¬ 
scale (x axis) 

28 

3.56 

.694 

Table 2 : 

Absolute 
Knowledge 
Sub-scale (y 
axis) 

28 

2.78 

.633 

Table 3: Self¬ 
regulation 
Sub-scale (z 
axis) 

28 

2.81 

.519 


The third axis position, z2, indicates that the 
learner population experiences low to moderate 
self-regulation. Learners in this population are 
guided by moderate cognitive learning 
strategies, capable of learning in blended or 
hybrid approaches to instruction, and increasing 
levels of motivation to learn. To positively alter 
trajectory, increased meta-cognitive 

instructional strategies provide learners with a 
proven path or plan for how to learn, based on 
prior learning accomplishments. The z2 learner 
population can develop a diminished need for 
faculty in the learning process, demonstrate 
increased persistence toward difficult problems, 
and alter learning strategies in response to 
levels of success in meeting learning goals and 
objectives. This learner population also is 
characterized by increased self-awareness, 
higher levels of self-efficacy, and some ability to 
monitor, evaluate, and alter individual 
performance, initiative, time-on-task, and level 
of effort. 
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The student population is now defined such that 
design elements can be applied in the construct 
of a course to more fully engage and 
accommodate the learner population, or to 
develop a strategy to alter the trajectory of the 
learner population toward the optimal position of 
x3, y3, z3. Once the coordinate position of the 
learner population is determined, design 
elements (Tables 5, 6, and 7) with respect to 
student profile, pedagogical strategies, and 
assessment mechanisms can be applied via 
instructional design. 

Essentially, mapping this student population 
position within the epistemic beliefs framework 
(Figure 1) to course design elements (Tables 5, 
6, and 7) provides guidance for course design 
that most efficaciously meets the needs of the 
learner population. Moreover, as noted 
previously, the methodology can be used to 
establish a trajectory of design to move a given 
learner population from its defined position to a 
more optimal position within the framework. 

6. APPLYING RUBRICS IN INSTRUCTIONAL 
DESIGN 

The subject of this case study, a population 
(N=28) of first-year undergraduate students 
involved in a state university Information 
Systems program, was evaluated through 
survey item mean analysis to a coordinate 
position of x3, y2, z2 with respect to the 
framework for determination of learner 
epistemic beliefs profile (Figure 1). Applying the 
rubric to first identify the student population's 
epistemic belief profile (Table 5) suggests the 
class has emerged from dualistic to relativistic 
and self-affirmed tendencies, is capable of 
contextual and integrative problem solving, can 
appreciate multiple world views, and possesses 
a capacity for critical analysis. 

Moreover, the student population has emerging 
tendencies toward intrinsic motivation, has 
developed and somewhat embraced a tolerance 
for ambiguity, and has an emerging sense of the 
contextual nature of knowledge. Also, the 
student population most appreciates knowledge 
based on practical applications. Data analysis 
also indicates the student population evaluates 
at a low to moderate level of self-regulation, 
shows tendencies toward active learning, can 
differentiate between faculty dependent and 
student dependent learning, and is open to 
collaborative learning environments. 


Table 5: Student Epistemic Profile 


i 



Dualistic 

Binary problem solving 
Concrete world view 
Does not acknowledge muluple per¬ 
spectives 

No critical analysis 
Motivation is extrinsic 
Does not see the "big picture" 
Intolerant to ambiguity 


Knowledge is absolute 
Knowledge is based on scientific rea¬ 
soning and natural laws 


Low self-regulation 

Passive learner 

Faculty dependent learner 

Does not value collaborative learning 


Relativistic 

Contextual problem solving 

Appreciates multiple world views 
Capacity for critical analysis 

Motivation may be extnnsic/intnnsic 

Sees the "big picture" 

Tolerant to ambiguity 

Knowledge is relative 

Knowledge is based on practical ap¬ 
plications 

Moderate self-regulation 

Active/pas sive learner 

Faculty/student dependent learner 

Open to collaboration learning 

Self-affirmativecommitted 

Integrative problem solving 

Appreciates and incorporates multiple 
perspectives in the development 

Employs highly cognitive, critical 
thinking skills 

Motivation is intrinsic 

Sees self in the "big picture" 

Embraces ambiguity 

1 

Knowledge is constructed aid 

grounded in integrative, interdis¬ 
ciplinary perspectives 

3 

High self-regulation 

Active learner 

Student dependent learner 

Seeks opportunities for collaborative 
learning 


Source: Dr. Michael Flerndon, Virginia 
Polytechnic Institute and State University 


Applying the rubric to identify applicable 
pedagogical strategies (Table 6) suggests 
initiation of some class discussion with an 
encouraging tone for students to participate in 
the discussions. Students should be encouraged 
to contribute to the base of knowledge and 
faculty should develop blended approaches for 
the dissemination of knowledge, such as a 
mixture of face-to-face instruction with online 
instruction. Moreover, faculty should utilize 
moderate cognitive learning strategies and 
develop assignments to diminish faculty 
responsibility for learning. 


Table 6: Pedagogical Strategies 

X Axis 

Ferry Model 

Y Mis 

Theory of Knowledge 

7. Axis Self Regulation 

1 

Faculty centered lectures 

Little to no class discussion 

1 

The faculty are the source of knowledge 
Synchronous dissemination of knowledge 
Controlled by faculty 

Utilize low cognitive learning strategies 
Design only in-class learning opportuni¬ 
ties 

Avoid an openness to change in instruc¬ 
tion 

Initiate some class discussion 
Encourage students to partici¬ 
pate in class discussions 

Allows faculty and students to contribute 
to llie base of knowledge 

Develop blended ^iproaches for the dis¬ 
semination of knowledge (e.g. a mix¬ 
ture of online and lace to face instruc¬ 
tion) 

2 

Utilize moderate cognitive learning 
strategies 

Develop assignments to diminish faculty 
responsibility for learning 

3 

Foster robust class discus- 

Em power students with a 
large degree of owner¬ 
ship for the success of the 
class 

3 

The construction of knowledge is student 

Asynchronous dissemination of knowl¬ 
edge 

Create knowledge through peer learning 
(e.g.. discuarion boards, group pro¬ 
jects. and other collaborative experi- 

3 

Implement meta-cognilive learning 

Encourage both in and out-of-class learn¬ 
ing opportunities 

Develop assignments that facilitate the 
adaptability to change in instruction 


Source: Dr. Michael Flerndon, Virginia 
Polytechnic Institute and State University 


Applying the rubric to determine the most 
appropriate assessment mechanisms (Table 7) 
for the student population indicates use of 
multiple choice test items. Student populations, 
as in this case, that have emerged from 
dualistic, binary thinking to relativistic in-context 
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thinking can relate to multiple choice test items 
where relativistic thinking is assessed. 

Table 7: Assessment Mechanisms 



Source: Dr. Michael Herndon, Virginia 
Polytechnic Institute and State University 

Also, assessment tools and mechanisms should 
encourage some degree of critical thinking and 
synthesis to support an emerging sense of the 
contextual nature of knowledge. Moreover, this 
student population should take some 
responsibility for gauging individual progress 
through assessment mechanisms such as 
portfolio assessment, reflection and self- 
assessment, and comparative evaluations to 
one's peers. 

7. FINDINGS AND RECOMMENDATIONS 

This baseline study establishes a priori 
knowledge regarding the level of epistemic belief 
among a cross-sample of undergraduate 
Information Systems students. As seen in Figure 
4, the student population demonstrates a 
moderate level of epistemic beliefs with respect 
to perceptions of knowledge and knowing (Ql- 
Q13), simple to moderate levels of epistemic 
beliefs with respect to self-regulation (Q14- 
Q32), and (increasing) moderate levels of 
epistemic beliefs with respect to self-efficacy 
(Q33-Q45). 

Additional research is needed to establish 
longitudinal views of Information Systems 
students by year in school with subsequent 
assessment data collected to establish pre and 
post programmatic results as an indicator of 
improved levels of epistemic belief. In this case 
study, the student population demonstrates 
tendencies toward a trajectory to increased 
levels of epistemic beliefs. 
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Figure 4: Item data graphed to illustrate levels 
of epistemic belief 

Other general findings in support of conclusions 
include the need to alter and support learning 
environments, strategically align pedagogical 
strategies, and employ more appropriate 
assessment mechanisms in response to the 
Information System student population's current 
levels of epistemic belief. For example, learners 
in a collaborative problem solving environment 
receive feedback and comments from peers, and 
from the teacher on the steps of planning, 
implementing, and executing problem solving 
processes rather than only receiving feedback 
from the teacher on their performance. 
Therefore, peer pressure, as a motivating factor, 
may lead students to perform higher level 
cognitive functions. In addition, social 
constructivism (Pajares, 2002) suggests that the 
exchange of critical feedback among peers as 
well as from the instructor can encourage 
students to modify their work. 

This study promoted the Scholarship of Teaching 
and Learning (SoTL) and extended the state of 
knowledge in Human Performance Technology 
by contributing to and exemplifying accepted 
learning theories and models. Students' 
perceptions of various aspects of teaching and 
learning in a course play an important role in 
their engagement and performance (Schommer, 
1993). Ravert and Evans (2007) showed that 
expecting students at earlier stages of 
development to learn from courses based on 
principles of negotiation, shared construction, 
and peer-to-peer learning could be problematic. 
Therefore, if tools employed in teaching and 
learning or instructional design run contrary to 
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students' epistemic beliefs, the result could be 
student frustration and distress. As a result, the 
instructional design and pedagogical strategy 
should address these issues during the course 
design phase. 

In this report a study examining student 
epistemic beliefs was presented. The researchers 
offer the following suggestions for further 
discovery to faculty, instructional designers, and 
administrators who develop curricula for 
undergraduate Information Systems students 
entering college with undetermined levels of 
epistemic belief: 

i. The authors suggest that faculty 
consider the use of epistemic belief data 
when developing course syllabi. 
Instructors should determine if the 
course design is structured in such a 
way to challenge and positively alter 
students' epistemic beliefs or only 
reinforce current epistemic beliefs. 

ii. Epistemic beliefs among students were 
discussed in this research; however, 
examining the influence of faculty 
members' epistemic beliefs on students' 
epistemic beliefs is fertile ground for 
future research endeavors. Little to no 
scholarship has been devoted to this line 
of inquiry. 

iii. This case study involved students in 
undergraduate Information Systems 
studies. The quantitative findings of this 
research may be generalized to students 
in multiple disciplines and year of study, 
as they relate to epistemic beliefs. 
Follow up study is needed to apply this 
methodology to broader boundaries. 

iv. Faculty should apply the rubrics for 
student epistemic profile, pedagogy, and 
assessment in support of instructional 
design for Information Systems courses. 

v. Research should further compare and 
study epistemic beliefs across 
disciplinary boundaries. The results will 
inform new efforts and planning phases 
in instructional design and curricula 
quality improvement initiatives. 

vi. While the authors used one proven 
instrument to assess the epistemic 
beliefs in this case study, multiple tools 
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exist. Course developers should choose 
an instrument that is most appropriate 
for their population and then apply the 
findings as was done in this case study. 

vii. Finally, the authors suggest that the 
study of epistemic belief should occur in 
a longitudinal fashion. Institutions can 
gauge students' epistemic beliefs at the 
beginning of their first year and 
periodically assess shifts and trends 
among students throughout the 
undergraduate experience. This process 
can allow faculty members to fine tune 
course design, academic activities and 
assignments, and course assessments, 
promoting growth in academic 
performance among their students. 
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